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SUMMARY 


In this paper a model is described of the human observer and decision 
maker monitoring a dynamic process. The decision process is described in 
terms of classical sequential decision theory by considering the hypothesis 
that an abnormal condition has occurred by means of a generalized likelihood 
ratio test. For tnis, a sufficient statistic is provided by the innovation 
sequence which is the result of the perception and information processing 
submodel of the human observer. On the basis of only two model parameters 
the model predicts the decision speed/accuracy trade-off and various atten- 
tional characteristics. 

A preliminary test of the model for single variable failure detection 
tasks resulted in a very good fit of the experimental data. 

In a formal validation program a variety of multivariable failure detection 
tasks was investigated. The task variables were the number, the bandwidth 
and the mutual correlation of display variables and various failure charac- 
teristics. 

A very good overall agreement between the model and experimental results 
showed the predict i\e capability of the model. In addition, the specific 
effect of almost all task variables was accurately predicted by the model. 


INTRODUCTION 


With increasing complexity and automation of man-machine systems the 
human operator’s role shifts from controller to supervisor. In the context of 
transport aircraft operation this is very much the case after the introduc- 
tion of automatic approach and landing systems and the future microwave 
landing system (MLS). 

The last two decades considerable research effort has been devoted to the 
study of humaui control behavior. One result is a number of mathematical 
tools, of which the state-space, time-domain optimal control model has been 
shown to provide a general framework adequately describing the human pro- 
cessing of information provided by a dynamic system (Refs 1-U). This can be 
extended to other cognitive functions involved in monitoring an automatic 
system, detecting system failures, making decisions, etc. The insight in 
this higher mental functioning is still rather incomplete although some 
attempts have been made to investigate and to model failure detection and 
simple decision making behavior (Refs 5-9)* 

This paper summarizes the results of a theoretical and experimental 
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analysis of the humeu^ observer and decision maker in multivariable failure 
detection tasks. In the next section a model of the human decision maker is 
formulated in terms of multivariable classical sequential decision theory, 
accounting for the important effect of correlated information. In the subse- 
quent section the model is tested against the results of a single variable 
task experiment reported in reference 8. Next, a formal model validation 
experiment is discussed. The latter results are extensively presented in 
reference 10. 


MODEL OF THE HUMAN OBSERVER AND DECISION MAKER 


It is assumed that the human perceives information of a linear dynamic 
system which is described by a Gauss-Markov random sequence. Based on the 
known dynamics of this system and the perceived information (i.e. noisy 
observations), the human makes the best estimate of the system state. Tnis 
is described in standard linear estimation theoretical terms (Kalman filter. 
Kefs 11-12) and is part of the well documented optimal control model. (Refs 
1-2) but included in fiffire 1 for the ease of reference. 

Now, in the normeil mode of operation, the discrepancy between perceived 
and expected information (the so-called innovation sequence nj^) is a zero 
mean Gaussian purely random sequence (Ref. 12) with covariance N^. It is 
assumed that abnormal system operation, as caused by errors in display 
instruments, malfunctioning of the system and excessive system disturbance 
levels (e.g. , large windshears in aircraft operation) can be represented by 
a deterministic process, as such unknown to the human observer but detected 
on the basis of a non-zero mean innovation sequence whose statistic is 
sufficient to make decisions (test hypotheses) when the system is completely 
observabl e. 

In terms of classical sequential decision theory (Refs 13-lM a so- 
called generalized likelihood ratio test can be formulated. The test eunounts 
to the comparison of the probability of a non-zero mean with the probability 
of a zero mean innovation sequence assuming tliat the human operator makes a 
short-term estimate of the mean of the inu.y.''ation sequence on the basis of 
the sample mean of m past observations (ft^^ in figure l). 

It can be derived (Ref. IS) that the effect of each observation at stage k on 
t.he (log of the) likelihood ratio is given by 

■' ''ll < ’ > 

under t'''"' assumption that the sample mean is const .uit during m observa- 
tions. The accvunulating effect of each observation on the total (log of the) 
likellnood ratio is given by the recursive expression 

h- h-. * “K <=> 

assuming that the innovation sequence is a white noise sequence (independent 
samples) which is exact in the normal mode of operation. Ttie number of obser- 
vations, based on which the decision is made, is chosen such that AL. is, on 
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the average, not decreasing; in other words, using only confirming evidence 
to make the decision. Otherwise, the positive semi-definite elements AL^ 
would have an accumulating effect on the likelihood ratio. 

When the likelihood ratio K (representing the total evidence of abnormal 
system operation) is equal to, or larger than, a decision threshold T, the 
decision is made that an abnormal condition has occurred. This decision 
threshold can conveniently be related to the accepted (or assumed) risk 
according to (Ref. ^k) T=( 1-Pj.)/P„, with Pj. the miss probability (i.e. of no 
response to an abnormal condition; and P^, ^e false alarm probability. 

Following reference an expression can be derived (Ref. 15) for the 
average number of samples used to make the decision that the system is ope- 
rating abnormally, which is, for a given sample rate, uniquely (linearly) 
related to the average detection time. 

This average number of observations K, given the abnormal condition, is 
given by 

2P^T^nT 


E{h’N'" ft} 

where (.) indicates the average over the ensemble and E{(.)} is the average 
over the sequence. 

Equation (3) gives a relationship between the average number of observations 
and the decision error probabilities (P^^ and 9 for a given innovation 
covariance N and the non-zero mean failure state sequence which is, however, 
a given task variable. Thus the only human decision model parameters are the 
short-term average sample size m and the innovation covariances which depend 
exclusively on the huma'i observation noise covariances (Ref. 15). The model 
output is the average failure detection time corresponding with given (or 
assumed) error probabilities. 

Previous studies (Ref, 2) support the hypothesis that the observation 
noise covariance scales with the mean-squared value of the corresponding 
signal. Thus for display variable j V‘.= P^^E(y?}, j= 1,....^. P^ represents a 
nominal, "f\ill attention" noise ratio'^ ( typicaily O.OItt). In case a display 
variable is not looked at (foveally), it is assumed that the corresponding 
observa\.ion noise is infinite, thus neglecting peripheral viewing. Inserting 
this expression in equation (3) results, after some matrix manipulation 
(Ref. is) in a very simple express io!i for the average "detection time" 

2P TinT 

(1,^) 

E fh-} 

L 

with 

('-Pj. ) (^b) 

‘J j .) 

is tlu? ''t’feotivo, rt'lat.i.ve, est.imatfd sample nnvui-square ; the subscript r in- 
dicat the normal i sat i’>n by the (constant) observation noise covariance V., 
p. is the estimation error covariance of display variable j, and indica^,es 
the average over the ensemble, and the sequence, and the display variables. 
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The expected value of fi^ over the display variables involves the probability 
distribution of the human attention to these variables. When an optimal dis- 
tribution (i.e., yielding a minimal detection time) of the human attention 
is assumed, the only remaining human decision model parameters are the short- 
term average sample size m and the overall level of attention Pq, 

The general model structure (the only assumptions are that the dynamic 
sy im is linear and that abnormal conditions can be represented by a deter- 
ministic process) accounts for the effect of a variety of task variables such 
as the number and bandwidth of display variables, the correlation among them 
and the perceived failure characteristics. These variables are included in 
the selected task configurations investigated in the validation program dis- 
cussed in the next section. 


MODEL VALIDATION 


In order to test the validity of the model the results of two experi- 
mental programs were considered and compared with the model predictions. The 
first experiment is reported in reference 8 in which obseirvers were required 
to detect a change in the mean of a stationary stochastic process. The exper- 
imental results were used for a preliminary validation of the human decision 
model and to "calibrate” the model with respect to the short-term average 
sample size m. Next, a formal model validation program is described which is 
reported in reference 15. 


Preliminary model validation 

In the first experiment the (two) subjects were instructed to detect, as 
soon as possible, the occurrence of a non-zero mean component while observing 
a second order, zero mean process. Both step and ramp failures with four 
possible amplitudes were introduced yielding 8 experimental conditions. 

Model results of the average failure detection time were obtained assuming 
the decision error probabilities of 0,05 which were also obtained in the 
experiment and on the basis of a typical overall level of attention P of 
O.OItt. The remaining model parameter ra was selected so as to obtain the best 
overall match with the experimental results. As shown in figure 2 the resul- 
ting value of m of four seconds yields an excellent fit to the experimental 
data. This can be expressed in the linear correlation coefficient between 
model and experimental results of 0.99 and in the ratio of model detection 
times and corresponding experimental values (t /t ), This ratio was on the 
average, 0,98, with a standard deviation of 0,99.^Thus the constant vadue of 
m of U seconds yielding a good fit for all the experimental conditions seems 
a human operator-related parameter. This value of four seconds, which ties in 
very well with the short-term memory spam typically reinging up to five seconds 
for viiiual stimuli (Ref. l6), will be assumed auid kept constant in following 
validation experiment. 
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Formal model validation 


A variety of tasks was specified so as to present the most cnacial com- 
bination of the task variables considered: nximber, bandwidth eind mutual 
correlation of the displays and failure characteristics. Two failure types 
were investigated either appearing on one display (display failure, F^) or 
on two correlated displays (system failure, Also the effect of prior 

knowledge about the failure type was investigated. 

Up to four-display tasks were considered consisting of two separate (inde- 
pendent) identical processes. Each process coiild be observed via two dis- 
plays: a relatively high bandwidth variable y1 (second order process with a 
break-frequency of 1.2 rad/s) was additionally filtered (first order filter 
with a time constant of four s<iconds). The output of this filter which is 
correlated with y1 (r - 0.5) wus displayed as y2. This process was duplica- 
ted resulting in a four-display pi'ocess (y1 to yU), of which the displays 
were two by two correlated. 

The resulting 8 configurations and display situation are summarized in table 
1. The configurations were inv'>stLgaled for two failure rates (a ramp with 
'i slope of 0.1 standard deviation of the display position per second and a 
slope of 0.2 0 /s) resulting .n l6 experimental conditions. A system failure 

^i . . » 

appealed as a ramp on yl which was additionally filtered and subsequently 
displayed. For a detailed presentation of the foregoing tasks and the model 
and experimental results (of three subjects, being general aviation pilots; 
twelve replications per condition), the reader is referred to reference 15. 

In this paper only the principal results are summarized aimed at a test of 
the human decision model. 


Failure detection times 

Model predictions of the (ensemble) mean failure detection times for all 
the U) failure detection tasks were obtained on the basis of the two constant 
model parameters: the overall level of attention C.OItt emd the short- 
term average sample size m = U seconds. Based on the previous results a value 
of the false alarm probability of 0.05 assumed. Additional model assump- 
tions and procedural details are discussed in reference 15 - 
The results of two subjects achieving an overall false alarm probability P^, 
of 0.0"^ could be compared directly with the model predictions. The result is 
summarized in figure 3^ The linear correlation coefficient between the model 
predictions and the experimental mean failure detection times is 0.86 which 
reflects a very good overall predictive capability of the human decision 
model. The ratio of the model and experimental failure detection time t /t 
is an ot!)cr measure for the agreement between the model and experimentaT ^ 
results. On the average (over all 16 tasks) this ratio is 0.98. The standard 
deviation is 0.12 which is comparable with the reliability of the experi- 
mental t’stimated means. 

This reliability of the data is included in figure U showing the model and 
oxp<?rimetital lailure detection times per configuration. Apart from the mean 
value, valso the standard deviation of the mean value estimate (a/^N, with a 
the standard deviation of the raw data) is given. For almost all configurations 
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the model predictions agree rather well with the experimental failure 
detection times. 

For one configuration (Conf. U) the model predictions clearly disagree with 
the experimental results. A very plausible explanation of this discrepancy 
(which was confirmed during the debriefing of the subjects) was that the 
subjects did not realize (use) the system failure dynamics but assxuned that 
the system failure appeared simultaneously on both (correlated) displays. 

The model results based on this assumption are also shown in figure 3 and 
indicated with model refinement. In that case, the linear correlation coeffi- 
cient is 0.95; the meeui ratio t /t is 1.01 with a standard deviation of 0.09. 

m e 

The agreement between the model predictions and experimental resu^'ts 
with respect to the specific task variables is summarized in table 2. The 
configurations involved in the pair-wise comparison between the configura- 
tions which differ with respect to the specific task variable (only) are 
indicated. 

Comparing the measured and model results shows that the effect of display 
bandwidth, of additional (correlated) displays and of the failure rate is 
excellently predicted by the model. The predicted interference between 
uncorrelated displays (because of the human attention sharing involved) is 
larger than obtained experimentally. The model predictions are based on a 
constant level of attention. However, the physiological (heart rate) meas- 
ures obtained during the experiment suggest a small, but statistically sig- 
nificant, increase in attention with an increase in displays which can 
easily explain the small difference in interference. The effect of the 
failure type is discussed before. It can be seen that the model refinement 
and the experimental results agree closely. 

The experimental results of the third subject reflect a distinctly different 
decision strategy. He made no false alarms and his failure detection times 
were, on the average, Uo ? higher. Yet his results correlated well with the 
model i'redictions (r = 0.80). Model results based on a very low value of P 
yield an arbitrary good overall correspondance with the measured failure ^ 
detection times (detection times increase monotonically with decreasing P ). 
However, virtually the same linear correlation coefficient is obtained. As 
discussed in reference 15 this signifies that the predicted effect of the 
various configurations on the failure detection time match the results of 
thio ntibject with an accuracy of about 12 % (the standard deviation of t /t 
is 0.12), ™ e 


Scanning behavior 

Various attentional characteristics can be derived from the foregoing 
model of the human observer and compared with eye scanning data obtained 
in the experimental program. 

Combining eqs (l), (3) and (U), it can easily be seen that the (ensemble) 
average effect on the likelihood ratio of one observation of display varia- 
ble j is given by 
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where 6. indicates the probahility of attending to j. In the normal mode 
of operation (n= 0) an alternative expression can he obtained (Ref. 15) 

normed 

dL. = 5 Ap . 3. (5b) 

J mode ^ 

where is the reduction in the estimation error covariance due to 

observing'^variable j. 

Equations (U) and (5) show that the optimal allocation of attention 
among the displayed variables (i,e. yielding the meocimum AL and thus, the 
minimal detection time) is obtained for the maximum expected value of fi . 

The model predicts (Ref. 15) that the optimal fraction of attention to the 
high bandwidth display(s) is varying between O.k and 0.7 (the total atten- 
tion to the high and low bandwidth display(s) is 1.0) somewhat depending on 
task specifics and failure characteristics. As the failure detection time 
is relatively insensitive to the division of attention in this region, a 
relatively constant fraction of attention to the high bandwidth display! s) 
for all configurations is predicted by the model, say between 0.5 and 0.6 
(enhanced by the randomized block design and the corresponding transfer of 
training). This agrees vei*y well with the experimental dwell fractions. The 
average dwell fraction on the upper display was not depending on the confi- 
gurations and varied between 0.5 and 0.55- 

The optimal allocation of attention in the time domain - thus the opti- 
mal scanning strategy for a given task - is described by eq. (5). Various 
model predictions can be derived from the attention allocation model and 
compared with the corresponding eye scanning measures. This is discussed in 
reference 15. For illustrative purposes, consider the normal mode of system 
operation, for which situation the effect of observing is described by eq. 
(5b). The model predicts that there is a scanning preference for high band- 
width display variables as the estimation error covariance Ap increases with 
display frequency, for a given amount of time. Furthermore, it follows from 
eq. (5b) that tnere is a scanning preference for correlated display varia- 
bles, because observing variable j yields an additional reduction in Ap. if 
variable j is correlated with i which results in a maximxim total AL and a 
minimum (average) failure detection time. 

These model predictions can be compared with the experimental eye scanning 
data in terms of display link values. The experimental results agreed well 
with these qualitative predictions. The fraction of links between the corre- 
lated displays was about two times the fraction of links between the uncor- 
related displays (0.6^ versus 0.36; in the failure mode this division is 
0.70 versus 0.30 which increase is predicted on the basis of eq. (5a) as 
discussed in reference 1*^). Furthermore, the fraction of scans (number of 
observations) towards the high bandwidth displays was two times the fraction 
of scans towards the low bandwidth displays. 

The foregoing analysis illustrates the predictive capability of the attention 
allocation model which may be a powerful tool in the study of human informa- 
tion processing tasks and display design problems. 
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CONCLUDING REMARKS 


In this paper a model of the humein observer and decision maker is 
summarized. The model consists of two parts. A submodel of the human obser- 
ver is formulated in linear estimation theoretical terms including the per- 
ception of the displayed information of a linear(ized) process and the in- 
formation processing stage which is described by a Kalman filter. The resul- 
ting innovation sequence provides a sufficient statistic for the decision 
process. In terms of classical sequential decision theory the hypothesis is 
considered that an abnormeil condition has occurred by means of a likelihood 
ratio test. An abnormal condition is represented by a deterministic pirocess 
which has to be detected on the basis of noisy observations of a normally 
zero-mean stochastic process. On the basis of only two model parameters 
(short-term average sample size m and the overall level of attention Pq) the 
model predicts the (ensemble) mean failure detection time and various atten- 
tional characteristics. 

A preliminary test of the model for single variable failure detection 
tasks resulted in a very good fit of the experimental results for a constant 
value of m of four seconds. 

This constant value for m and a typical value for the overall level of atten- 
tion were used to predict the mean failure detection times and scanning 
characteristics of a variety of multivariable failure detection tasks which 
were investigated in the formal validation program. The task variables were 
the number, the bandwidth and the mutual correlation of display variables 
and various failure characteristics. 

A very good overall agreement between the model and experimental results both 
in terms of failure detection times and eye scanning measures showed the 
predictive capability of the model. In addition, the specific effect of al- 
most all task variables was accurately predicted by the model. 


REFERENCES 


1. KLeinman, D.L. and Baron, S.: Manned Vehicle System Analysis by meaiis 
of Modern Control Theory. NASA CR-1753, June 1971. 

;l. Baron, S. and Levison, W.H.: Display Analysis with the Optimal Control 
Model of the Human Operator. Human Factors, 1977, 19(5). 

1. Wewerinke, P.H.: Performance and Workload Analysis of In-Flight Heli- 
copter Missions. Paper presented at the 13th Annual Conference on Menual 
Control, MIT, June 1977 (also NLR MP 77013 U). 

i*. Wewerinke, P.H.: The Effect of Visual Information on the Manual Approach 
and sanding. Paper presented at the l6th Annual Conference on Manual 
Coft.rol, MIT, May 1080 (also NLR MP 80019 U). 


- 564 - 



1'* -r 


5. Phatak, A.V. and Kleinman, P.L.: Current Status of Models for the Human 
Operator as a Controller and Decision Maker in Manned Aerospace Systems. 
AGARD symp. on Automation in Manned Aerospace Systems. WPAFB, Dayton, 

USA, 1972. 

6. Levison, W.H. and Tanner, R.B.: A Control-Theory Model for Human Decision 
Making. NASA CR-1953, December 1971. 

7. Wewerinke, P.H. : Human Monitoring and Control Behavior - Models and Expe- 
riments. NLR TR 77010 U, October 1976. 

8. Gai, E.G. and Curry, R.E.: A Model of the human Observer in Failure 
Detection Tasks. IEEE Trans, on Systems, Man and Cybernetics. Vol. SMC-6 
no. 2, February 1976. 

9. Gai, E.G. and Curry, R.E.: Failure Detection by Pilots during Automatic 
Landing: Models and Experiments. J. Aircraft, Vol. lL, no. 2, February 1977- 

10. v.d. Graaff, R.C.: An Experimental Analysis of Human Monitoring Behavior 
in Multivariable Failure Detection Tasks. 

NLR TR 8l06'^ U, June 1931. 

11. Bryson, A.E. and Ho, Y.C.: Applied Optimal Control. Blaisdell Publishing 
Company, 1969. 

12. Mehra, R.K. and Peshon, J. : An Innovation Approach to Fault Detection and 
Diagnosis in Dynamic systems. Automatics, Vol. 7, 1971. 

13. Wald, A.: Sequential Analysis, J. Wiley, New York, 19**7. 

1U. Sage, A.P. and Melsa, J.L.: Estimation Theory with Applications to Commu- 
nications and Control. McGraw-Hill, 1971. 

15. Wewerinke, P.H.: A Model of the Human Decision Maker Observing a Dynamic 
System. NLR TR 81062 U, March I98I. 

16. Sheridan, T.3. and Ferrell, W.R.: Man-Machine Systems: Information, Control 
and Decision Models of Human Performance. MIT Press, Cambridge, Ma. 197l». 


- 565 - 


r 



Table 1a 

Tsisk configurationa 


Table lb 

Display situation 


CONF. 

DISPLAY 

FAILURE 

1 

yi 


2 

y2 

^2 

3 




D 


yi, y2 

F 

s 

5 



F^, or Fg, or F^ 

6 

yi, y3 

P^ or F^ 

7 


— 


8 

yi. 

y2, y3, yi* 

F. , or F 
1 .s . 

J 


Table 2 

Specific effect of task variables on model and 
experimental failure detection cimes 


EFFECT OF CONFIGURATIONS 


bandwidth 

(low/high) 1. 2, 5. 8 


addit ional 

display 1, 3, 6, 7 
info. 


inter 
ere 
failure 
type 

{ system/ 
display) 
failure 
rate 

( hieh/low) 


1, 3, 5, 


3, k, 8 


RATIO t./t 


measured 


B m 


1.01IC.12I 1.00 0.0! 


| 0 . 76 | 0 . 06 | 0.77 


1.11 0.051 1.2 


1.1*310.331 1.21* 0.06 

(1.37) (0.21) 


6710.051 0.69 


(.) model refinement 
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flf. 1 Model of the huasn decieion aeker observing • dynuilc process 





FAILURE DETECTION TIME , (») FAILURE DETECTION TIME (*) 


20 r 



0) STEP INPUT yj. « cjay 


nr* 



“U 


T ST. DEV. 


1 


(S MEAN 

: MEASURED 



• 

: MODEL 





J 






X 


0 


JL 

1 


1 


1 , i 

2 


J L 


3 



Flf. 2 Exp«rlMatal and aodel failure datection tiiMa for various 
step and raap Inputs 
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MODEL PREDICTION, [s] 


Fig. 3 Overall comparison of model predictions and experimental 
results of two subjects 
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